System For Haptically Representing Sensor Input

ABSTRACT

A haptic representation system is provided that generates a haptic effect in response to sensor input. The sensor input is mapped to a haptic signal. The haptic signal is sent to an actuator configured to receive the haptic signal. The actuator utilizes the haptic signal to generate the haptic effect.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/816,336 filed on Nov. 17, 2017, which is acontinuation of U.S. patent application Ser. No. 15/347,073 filed onNov. 9, 2016 (now U.S. Pat. No. 9,846,485 issued on Dec. 19, 2017),which is a continuation of U.S. patent application Ser. No. 14/819,880filed on Aug. 6, 2015 (now U.S. Pat. No. 9,501,149 issued on Nov. 22,2016, which is a continuation application of U.S. patent applicationSer. No. 13/597,300 filed on Aug. 29, 2012 (now U.S. Pat. No. 9,116,546issued on Aug. 25, 2015). The specifications of each of theseapplications are herein incorporated by reference in their entireties.

FIELD

One embodiment is directed generally to a device, and more particularly,to a device that produces haptic effects.

BACKGROUND

Haptics is a tactile and force feedback technology that takes advantageof a user's sense of touch by applying haptic feedback effects (i.e.,“haptic effects”), such as forces, vibrations, and deformations, to theuser. A device, such as a mobile device, touchscreen device, andpersonal computer, can be configured to generate haptic effects in orderto provide a more immersive experience for the user. For example, when auser interacts with the device using, for example, a button,touchscreen, lever, joystick, wheel, or some other control, a hapticsignal can be generated, where the haptic signal causes the device toproduce an appropriate haptic effect. The user can experience the hapticeffect, and the user's interaction with the device can be enhanced.

SUMMARY

One embodiment is a haptic representation system that generates a hapticeffect. The system receives input from a sensor. The system maps thereceived input to a haptic signal. The system further sends the hapticsignal to an actuator to generate the haptic effect.

BRIEF DESCRIPTION OF THE DRAWINGS

Further embodiments, details, advantages, and modifications will becomeapparent from the following detailed description of the preferredembodiments, which is to be taken in conjunction with the accompanyingdrawings.

FIG. 1 illustrates a block diagram of a haptic representation system inaccordance with one embodiment of the invention.

FIG. 2 illustrates an example of a device with sensor input and hapticoutput, according to one embodiment of the invention.

FIG. 3 illustrates a block diagram of a device capable of providingvibrotactile effects and/or kinesthetic effects, according to oneembodiment of the invention.

FIG. 4 illustrates a diagram of an example mapping of sensor input to anoutput signal that, when played at an output device, produces an outputeffect, according to one embodiment of the invention.

FIG. 5 illustrates a flow diagram of the functionality of a hapticrepresentation module, according to one embodiment of the invention.

FIG. 6 illustrates a block diagram of a device capable of producing ahaptic effect to augment sensory perception, according to one embodimentof the invention.

FIG. 7 illustrates a block diagram of a device capable of producing ahaptic effect to augment visual information displayed within a displayscreen, according to one embodiment of the invention.

FIG. 8 illustrates a diagram of an example mapping of sensor input to aforce haptic signal that, when played at an actuator, produces a forcehaptic effect, according to one embodiment of the invention.

FIG. 9 illustrates a diagram of an example mapping of sensor input to adeformation haptic signal that, when played at an actuator, produces adeformation haptic effect, according to one embodiment of the invention.

FIG. 10 illustrates a diagram of an example mapping of sensor input to aimpedance haptic signal that, when played at an actuator, produces animpedance haptic effect, according to one embodiment of the invention.

FIG. 11 illustrates a flow diagram of the functionality of a hapticrepresentation module, according to another embodiment of the invention.

DETAILED DESCRIPTION

One embodiment is a haptic representation system that can produce ahaptic effect that represents input that is generated by a sensor andreceived by the haptic representation system. In certain embodiments,the received input includes extra-sensory information, whereextra-sensory information is information that cannot normally beperceived by a human being, such as an electromagnetic field, infra-redlight, ultraviolet light, radiation, or subtle changes in temperature.In some embodiments, the haptic effect is a deformation haptic effectproduced by a deformation actuator, where a deformation haptic effectincludes an alteration of a shape and/or size of a component operablycoupled to the deformation actuator. In certain embodiments, thedeformation haptic effect that is produced by the deformation actuatoris generated based on the extra-sensory information included in thereceived input generated by the sensor.

As described below in greater detail, a “vibration effect” or “vibrationhaptic effect” is an effect that produces a vibration. A “vibrationactuator” is an actuator configured to produce a vibration hapticeffect. A “force effect” or “force haptic effect” is an effect thatproduces a force. A “force actuator” is an actuator configured toproduce a force haptic effect. A “deformation effect” or “deformationhaptic effect” is an effect that deforms a structure or shape. A“deformation actuator” is an actuator configured to produce adeformation haptic effect. An “impedance effect” or “impedance hapticeffect” is an effect that produces resistance or opposition to anapplied force. An “impedance actuator” is an actuator configured toproduce an impedance haptic effect.

FIG. 1 illustrates a block diagram of a haptic representation system 10in accordance with one embodiment of the invention. In one embodiment,system 10 is part of a device (such as device 200 illustrated in FIG. 2,where device 200 is described below in greater detail), and system 10provides a haptic representation functionality for the device. Althoughshown as a single system, the functionality of system 10 can beimplemented as a distributed system. System 10 includes a bus 12 orother communication mechanism for communicating information, and aprocessor 22 coupled to bus 12 for processing information. Processor 22may be any type of general or specific purpose processor. System 10further includes a memory 14 for storing information and instructions tobe executed by processor 22. Memory 14 can be comprised of anycombination of random access memory (“RAM”), read only memory (“ROM”),static storage such as a magnetic or optical disk, or any other type ofcomputer-readable medium.

A computer-readable medium may be any available medium that can beaccessed by processor 22 and may include both a volatile and nonvolatilemedium, a removable and non-removable medium, a communication medium,and a storage medium. A communication medium may include computerreadable instructions, data structures, program modules or other data ina modulated data signal such as a carrier wave or other transportmechanism, and may include any other form of an information deliverymedium known in the art. A storage medium may include RAM, flash memory,ROM, erasable programmable read-only memory (“EPROM”), electricallyerasable programmable read-only memory (“EEPROM”), registers, hard disk,a removable disk, a compact disk read-only memory (“CD-ROM”), or anyother form of a storage medium known in the art.

In one embodiment, memory 14 stores software modules that providefunctionality when executed by processor 22. The modules include anoperating system 15 that provides operating system functionality forsystem 10, as well as the rest of a device in one embodiment. Themodules further include a haptic representation module 16 that generatesa haptic effect that represents received input, as disclosed in moredetail below. In certain embodiments, haptic representation module 16can comprise a plurality of modules, where each individual moduleprovides specific individual functionality for generating a hapticeffect that represents received input. System 10 will typically includeone or more additional application modules 18 to include additionalfunctionality, such as the Integrator® Haptic Development Platform byImmersion Corporation.

System 10, in embodiments that transmit and/or receive data from remotesources, further includes a communication device 20, such as a networkinterface card, to provide mobile wireless network communication, suchas infrared, radio, Wi-Fi, or cellular network communication. In otherembodiments, communication device 20 provides a wired networkconnection, such as an Ethernet connection or a modem.

Processor 22 is further coupled via bus 12 to a display 24, such as aLiquid Crystal Display (“LCD”), for displaying a graphicalrepresentation or user interface to a user. The display 24 may be atouch-sensitive input device, such as a touchscreen, configured to sendand receive signals from processor 22, and may be a multi-touchtouchscreen.

System 10, in one embodiment, further includes an actuator 26. Processor22 may transmit a haptic signal associated with a generated hapticeffect to actuator 26, which in turn outputs haptic effects, such asvibrotactile haptic effects and deformation haptic effects. Actuator 26includes an actuator drive circuit. Actuator 26 may be, for example, anelectric motor, an electro-magnetic actuator, a voice coil, a shapememory alloy, an electro-active polymer, a solenoid, an eccentricrotating mass motor (“ERM”), a linear resonant actuator (“LRA”), apiezoelectric actuator, a high bandwidth actuator, an electroactivepolymer (“EAP”) actuator, an electrostatic friction display, or anultrasonic vibration generator. In alternate embodiments, system 10 caninclude one or more additional actuators, in addition to actuator 26(not illustrated in FIG. 1). In other embodiments, a separate devicefrom system 10 includes an actuator that generates the haptic effects,and system 10 sends generated haptic effect signals to that devicethrough communication device 20.

System 10, in one embodiment, further includes a sensor 28. Sensor 28can be configured to detect a form of energy, or other physicalproperty, such as, but not limited to, acceleration, bio signals,distance, flow, force/pressure/strain/bend, humidity, linear position,orientation/inclination, radio frequency, rotary position, rotaryvelocity, manipulation of a switch, temperature, vibration, or visiblelight intensity. Sensor 28 can further be configured to convert thedetected energy, or other physical property, into an electrical signal,or any signal that represents virtual sensor information. Sensor 28 canbe any device, such as, but not limited to, an accelerometer, anelectrocardiogram, an electroencephalogram, an electromyograph, anelectrooculogram, an electropalatograph, a galvanic skin responsesensor, a capacitive sensor, a hall effect sensor, an infrared sensor,an ultrasonic sensor, a pressure sensor, a fiber optic sensor, a flexionsensor (or bend sensor), a force-sensitive resistor, a load cell, aLuSense CPS² 155, a miniature pressure transducer, a piezo sensor, astrain gage, a hygrometer, a linear position touch sensor, a linearpotentiometer (or slider), a linear variable differential transformer, acompass, an inclinometer, a magnetic tag (or radio frequencyidentification tag), a rotary encoder, a rotary potentiometer, agyroscope, an on-off switch, a temperature sensor (such as athermometer, thermocouple, resistance temperature detector, thermistor,or temperature-transducing integrated circuit), microphone, photometer,altimeter, bio monitor, or a light-dependent resistor.

FIG. 2 illustrates an example of a device 200 with sensor input andhaptic output, according to one embodiment of the invention. Aspreviously described, in one embodiment, device 200 can include a hapticrepresentation system (such as haptic representation system 10illustrated in FIG. 1) that can represent the sensor input as hapticoutput. According to the embodiment, device 200 is a mobile device thatcan be carried within a user's hand. Device 200 can include a sensor(such as sensor 28 of FIG. 1), where the sensor not illustrated in FIG.2, extensions 210 and 220, and display 230. The sensor can be configuredto detect sensory information and extra-sensory information, wheresensory information is information that can normally be perceived by ahuman being, and where extra-sensory information is information thatcannot normally be perceived by a human being. The sensor can be furtherconfigured to produce sensor input based on the detected information.Extensions 210 and 220 can each be coupled to an actuator (notillustrated in FIG. 2). The actuator can be a vibration actuator or adeformation actuator. A vibration actuator can cause extensions 210 and220 to vibrate based on the sensor input produced by the sensor. Adeformation actuator can cause extensions 210 and 220 to deform based onthe sensor input produced by the sensor. The specifics of a vibrationactuator and a deformation actuator are further described in greaterdetail. Display 230 can display visual information based on the sensorinput produced by the sensor.

FIG. 3 illustrates a block diagram of a device 300 capable of providingvibrotactile effects (also identified as “vibration haptic effects”)and/or kinesthetic effects (also identified as “deformation hapticeffects”), according to one embodiment of the invention. Device 300includes a housing 302, a display 304, a keypad 306, and extensions308-0 and 308-1. In another embodiment, keypad 306 is part of atouchscreen display 304. Device 300, in one embodiment, is a wirelessportable system capable of providing wireless audio/video communication,mobile data communication, a remote game console, and the like. Forexample, device 300 may be a cellular phone, a PDA, a smart phone, alaptop computer, a game console, and/or a handheld electronic devicecapable of processing information as well as providing haptic feedback.

To provide a haptic feedback to a user's hand in accordance with anoperation mode, device 300 is capable of macroscopically altering itsouter enclosure or housing 302 (which includes extensions 308) inresponse to the nature of the application. Depending on the application,extensions 308 can expand or contract (as indicated by arrows in FIG. 3)thereby macroscopically altering the shape and/or size of housing 302.In one embodiment, a shape is “macroscopically” altered if it changes tothe extent that the change can be detected by the user via, for example,sight or feel. For example, a cell phone or smart phone device capableof changing its outer enclosure shape (“shape changing device”) may beused to emulate a handshake between two users. To convey a handshake, afirst user, for instance, might squeeze its first shape changing deviceto cause a pulse or squeeze of a second shape changing device of asecond user, where the first and second users are engaged in acommunication, such as a telephone call, via the first and second shapechanging devices connected to the first shape changing device. In otherwords, a shape input or shape signal is sent from the first shapechanging device to a second shape changing device indicating that thesecond device should activate its haptic mechanism to change its shapefor emulating a handshake. In other embodiments, additional portions ofdevice 300 besides housing 302 may also change shape, such as display304, or input elements such as keypad 306.

Systems such as device 300 may employ vibrotactile effects (alsoidentified as vibration haptic effects) and/or kinesthetic effects (alsoidentified as deformation haptic effects) to emulate shape changingeffects. Vibrotactile effects, for instance, may be used to incorporatehaptic feedback to a user via a handheld device. Such haptic feedbackeffects may be characterized by relatively high-frequency (e.g., about160-220 Hz) and relatively small displacement (e.g., about 50-500micrometers) vibrations. Further, different types of haptic informationsuch as confirmation of button clicks and alerts can also be conveyed.Kinesthetic effects, on the other hand, may be characterized byrelatively large displacements (e.g., about 1-10 mm) and relativelylow-frequency (e.g., about 0-40 Hz) motions. Deformable or flexiblesurfaces can be used for effective emulation of kinesthetic effects,such as macroscopically changing surface properties depending on theapplication or activated feature.

Kinesthetic effects may be effectively emulated using deformable hapticsurfaces. For example, kinesthetic effects may allow a handheld deviceto be used as a directional navigation tool. In this example, activationof deformable surfaces at different locations on the handheld device canbe used as a haptic display of directional information. In anotherexample, kinesthetic effects allow performance of specific effects(e.g., pulsation, heartbeat, etc.), which could be of value in virtualtele-presence and/or social networking applications. In one example, aheartbeat of one person can be emulated by expanding and contractingdeformable pads on the sides of a cell phone of another person connectedvia a telephone call. In another example, a squeezing of a cell phone atone end of a call can be emulated as a handshake sensation at anothercell phone at the other end of the call.

Force haptic effects or “force effects” may be emulated using varioustypes of input signals to drive a haptic actuator, such as, but notlimited to, an ERM. Certain types of input signals may be used toprovide various impulse force effects or a “jerk sensation” as opposedto more constant force effects (e.g., pushing or pulling force effects).In one example, such impulse force effects may simulate being poked by afinger. In one example, such impulse force effects may simulate astrike, for example, of a golf club impacting a golf ball. In oneexample, such impulse force effects may simulate a racket impacting atennis ball. Impulse force effects may be used to simulate other gamingenvironments.

Device 300, in one embodiment, is able to change shape based on anoperating mode (e.g., application, activated feature, etc.), as opposedto merely being manipulated by a user. Various haptic materials and/oractuators can be used in the haptic mechanism to cause varying shapes ina flexible surface of device 300. For example, EAPs may be used to formone or more actuators in the haptic mechanism for shape changing basedon activation of control signals. In other embodiments, a piezoelectricelement, programmable gels, or a fiber of shape memory alloys (“SMAs”)can be used as actuators.

In one embodiment, indications of a device operating mode such as anactivated feature and application can activate predetermined patterns ofa haptic mechanism. Such patterns can then be applied to the flexiblesurface of device 300 using a deformation mechanism. A haptic substratethat includes a plurality of actuators can be applied to the surface toenact or form the patterns. EAPs, for example, can be employed to formone or more actuators in a haptic mechanism such that activating signalsreceived by the haptic mechanism can convey flexible surface shapes. Thehaptic substrate can be formed from micro-electro-mechanical systems(“MEMS”) elements, thermal fluid pockets, MEMS pumps, resonant devices,variable porosity membranes, laminar flow modulation, etc.

Extensions 308 can be controllable as to displacement, as well as anypulsation or other suitable effects and/or patterns. For example, oneuser can squeeze a first device, and a second device connected on a callto the first device can pulse or squeeze in the hand of a second user toconvey a physical handshake. Thus, a signal can be sent from the firstdevice to the second device to indicate that the second device shouldchange shape to emulate a handshake (e.g., a low frequency force orpressure like a squeeze of a hand). In this fashion, any predeterminedshape change characteristics or patterns supportable by the underlyinghaptic mechanism, substrate, and/or actuator control can be employed.

Further details of devices capable of producing shape changing ordeformation haptic effects are described in U.S. patent application Ser.No. 12/776,121, filed on May 7, 2010, herein incorporated by reference.

FIG. 4 illustrates a diagram of an example mapping of sensor input to anoutput signal that, when played at an output device, produces an outputeffect, according to one embodiment of the invention. More specifically,FIG. 4 illustrates sensor 400. Sensor 400 can be identical to sensor 28of FIG. 1. Sensor 400 can be a sensor configured to detect information.In certain embodiments, sensor 400 can be a sensor configured to detectextra-sensory information, where extra-sensory information isinformation that cannot normally be perceived by a human being. Forexample, in one embodiment, sensor 400 can be a magnetometer configuredto detect electromagnetic fields. In another embodiment, sensor 400 canbe a light sensor configured to detect light frequencies that are out ofrange of the human eye, such as an infra-red light frequency. In anotherembodiment, sensor 400 can be a temperature sensor configured to detecta change in temperature, even when the temperature change could notnormally be detected by a human being. Based on the detectedextra-sensory information, the sensor can generate sensor input, wherethe sensor input represents the detected extra-sensory information. Inother embodiments, sensor 400 can be a sensor configured to detectsensory information, where sensory information is information that cannormally be perceived by a human being.

FIG. 4 also illustrates representation module 410. When executed by aprocessor (not illustrated in FIG. 4), representation module 410 canreceive the sensor input from sensor 400, map the sensor input to anoutput signal, and send the output signal to output device 420.According to the embodiment, representation module 410 can include analgorithm that maps the sensor input to an output signal, where theoutput signal is configured to produce an output effect when played atoutput device 420. Thus, in certain embodiments, representation module410 can represent the extra-sensory information detected by sensor 400(and which cannot normally be perceived by a human being) as an outputeffect that can be played at output device 420 (and thus, can beperceived by a human being). In this manner, representation module 410can allow a human being to perceive extra-sensory information detectedby sensor 400, which normally could not be perceived. In otherembodiments, representation module 410 can also represent the sensoryinformation detected by sensor 400 as an output effect that can beplayed at output device 420. In this manner, representation module 410can augment a human being's normal perception of the sensory informationdetected by sensor 400.

In certain embodiments, representation module 410 can be a hapticrepresentation module, such as haptic representation module 16 ofFIG. 1. In these embodiments, when executed by a processor,representation module 410 can receive the sensor input from sensor 400,map the sensor input to a haptic signal, and send the haptic signal tooutput device 420, where output device 420 is an actuator, such asactuator 26 of FIG. 1. According to the embodiment, representationmodule 410 can include an algorithm that maps the sensor input to ahaptic signal, where the haptic signal is configured to produce a hapticeffect when played at output device 420. Thus, representation module 410can represent the extra-sensory information detected by sensor 400 (andwhich cannot normally be perceived by a human being) as a haptic effectthat can be played at output device 420 (and thus, can be perceived by ahuman being).

For example, in an embodiment where sensor 400 is a magnetometer,representation module 410 can map the input from the magnetometer into adeformation haptic signal or a vibration haptic signal for an actuator.A deformation haptic signal can be played at an actuator to generate adeformation haptic effect. A vibration haptic signal can be played at anactuator to generate a vibration haptic effect. As another example, inan embodiment where sensor 400 is a light sensor, representation module410 can map the input from the light sensor into a video signal that canbe displayed on a display screen, or an audio signal that can be playedat a speaker. As yet another example, in an embodiment where sensor 400is a radiation sensor, representation module 410 can map the input fromthe radiation sensor into either a deformation haptic signal or avibration haptic signal for an actuator.

In certain embodiments, sensor 400 can be local to representation module410 (i.e., sensor 400 and representation module 410 can be locatedwithin a single device). In other embodiments, sensor 400 is located ona separate device from representation module 410, and the sensor inputis sent to representation module 410 over a network.

In certain embodiments, the mapping performed at representation module410 includes determining whether the received sensor input exceeds aspecified threshold (i.e., “specified value”). If the received sensorinput exceeds the specified value, a corresponding output signal (suchas a haptic signal) is generated, and the received sensor input ismapped to the generated output signal. If the received sensor input doesnot exceed the specified value, no output signal is generated.

In alternate embodiments, the mapping performed at representation module410 includes mathematically transforming the received sensor input intoa corresponding output signal (such as a haptic signal). This can bedone continuously as sensor input is received, and thus, thecorresponding output signal can be continuously modulated. As the outputsignal is continuously modulated, the corresponding output effect (suchas a haptic effect) that is generated can also be continuouslymodulated.

FIG. 4 also illustrates output device 420. Output device 420 can receivethe output signal sent by representation module 410, and can generate anoutput effect based on the output signal. In certain embodiments, outputdevice 420 can be an actuator, such as actuator 26 of FIG. 1. In some ofthese embodiments, output device 420 can be a vibration actuatorconfigured to produce vibration haptic effects. In other embodiments,output device 420 can be a deformation actuator configured to producedeformation haptic effects. In these embodiments, output device 420 canreceive a haptic signal sent by representation module 410, and cangenerate a haptic effect based on the output signal. In embodimentswhere the haptic signal is a vibration haptic signal, output device 420can produce a vibration haptic effect. In embodiments where the hapticsignal is a deformation haptic signal, output device 420 can produce adeformation haptic effect. In other embodiments, output device 420 canbe an output device configured to produce video effects, such as adisplay screen, or an output device configured to produce audio effects,such as a speaker.

For example, in an embodiment where sensor 400 is a magnetometer, andoutput device 420 is an actuator (such as a vibration actuator or adeformation actuator), output device 420 can deform and/or vibrateaccording to a presence of an electromagnetic field, the intensity ofthe electromagnetic field, or other attributes of the electromagneticfield. More specifically, sensor 400 can detect the electromagneticfield and produce sensor input based on one or more attributes of theelectromagnetic field. Representation module 410 can map the sensorinput produced by sensor 400 to a vibration haptic signal, a deformationhaptic signal, or a combination therein. Based on the vibration hapticsignal, the deformation haptic signal, or the combination therein,output device 420 can produce a vibration haptic effect (by vibrating),a deformation haptic effect (by deforming), or a combination therein.

As another example, in an embodiment where sensor 400 is a light sensor,and output device 420 is a display screen, output device 420 can displaya visual representation based on the detection of infra-red light (orultraviolet light). More specifically, sensor 400 can detect theinfra-red light (or ultraviolet light) and produce senor input based oneor more attributes of the infra-red light (or ultraviolet light).Representation module 410 can map the sensor input produced by sensor400 to a video signal. Based on the video signal, output device 420 canproduce a video effect (e.g., a visual representation of the detectedlight). In an alternate embodiment, output device 420 can be an actuatorrather than a display screen. In this embodiment, representation module410 can map the sensor input produced by sensor 400 to a vibrationhaptic signal or a deformation haptic signal. Based on the vibrationhaptic signal, or the deformation haptic signal, output device canproduce a vibration haptic effect, or a deformation haptic effect, thatrepresents the detected light. For example, if sensor 400 detects asignificant amount of infra-red light, output device 420 can deform anamount that reflects the amount of detected infra-red light.

Thus, according to certain embodiments, a user's senses can beaugmented, as the user can perceive attributes of an environment thatwould normally be difficult or impossible to perceive.

FIG. 5 illustrates a flow diagram of the functionality of a hapticrepresentation module (such as haptic representation module 16 of FIG.1), according to one embodiment of the invention. In one embodiment, thefunctionality of FIG. 5, as well as the functionality of FIG. 11, whichis described below in greater detail, is implemented by software storedin memory or another computer-readable or tangible medium, and executedby a processor. In other embodiments, the functionality may be performedby hardware (e.g., through the use of an application specific integratedcircuit (“ASIC”), a programmable gate array (“PGA”), a fieldprogrammable gate array (“FPGA”), etc.), or any combination of hardwareand software. Furthermore, in alternate embodiments, the functionalitymay be performed by hardware using analog components.

The flow begins and proceeds to 510. At 510, input is received from asensor. The input can include extra-sensory information, whereextra-sensory information is information that cannot normally beperceived by a human being. In certain embodiments, the input caninclude one or more interaction parameters, where each interactionparameter can include a value. In certain embodiments, the sensor can bea magnetometer configured to generate the input based on anelectromagnetic field that the magnetometer detects. In otherembodiments, the sensor can be a light sensor configured to generate theinput based on energy that the light sensor detects. The energy caninclude ultraviolet light. Alternately, the energy can include infra-redlight. In other embodiments, the sensor can be a radiation sensorconfigured to generate the input based on radiation that the radiationsensor detector. The flow proceeds to 520.

At 520, the received input is mapped to a haptic signal. In certainembodiments, the mapping the received input to the haptic signalincludes generating the haptic signal when the value of at least oneinteraction parameter exceeds a specified value. In other embodiments,the mapping the received input to the haptic signal includescontinuously modulating the haptic signal based on a continuous updatingof the value of at least one interaction parameter. The flow proceeds to530.

At 530, the haptic signal is sent to an actuator to generate a hapticeffect. In certain embodiments, the actuator is a vibration actuator,and the haptic effect generated by the vibration actuator is a vibrationhaptic effect. In other embodiments, the actuator is a deformationactuator, and the haptic effect generated by the deformation actuator isa deformation haptic effect. The flow then ends.

In certain embodiments, a haptic device can output various types ofinformation based on sensor input. Examples include: outputting hapticeffects based on input from a magnetometer; outputting deformationhaptic effects on a display of the device to create a “geofence”;augmenting sensory perception by outputting deformation haptic effects;enhancing normal sensory perception by outputting deformation hapticeffects; and outputting haptic effects based on either a user's mood,ambient awareness, or bio feedback.

Furthermore, in certain embodiments, a deformation actuator of a hapticdevice can output deformation haptic effects using one of a plurality ofinteraction modes. Examples of the interaction modes include: applyingpressure to the body of a user; changing shape (where the shape-changecan include a change in a macro-shape of the haptic device or a changein a texture of the haptic device); and outputting an impedance hapticeffect (where the haptic device assesses user input and outputs one ormore haptic effects based on the user input).

FIG. 6 illustrates a block diagram of a device 600 capable of producinga haptic effect to augment sensory perception, according to oneembodiment of the invention. Device 600 can include sensor 610. Sensor610 can be identical to sensor 28 of FIG. 1. Sensor 610 can be a sensorconfigured to detect information. Based on the detected information, thesensor can generate sensor input, where the sensor input represents thedetected information. In certain embodiments, sensor 610 can be a sensorconfigured to detect extra-sensory information, where extra-sensoryinformation is information that cannot normally be perceived by a humanbeing.

Device 600 can further include a housing 620. Housing 620, alsoidentified as an outer enclosure of device 600, can be operativelycoupled to an actuator 630. Housing 620 can also include a hapticsurface which overlays at least a portion of housing 620. In certainembodiments, the haptic surface can be configured to apply a force inresponse to a force haptic effect produced by actuator 630. In theseembodiments, actuator 630 can be a force actuator that can be configuredto produce a force haptic effect. In other certain embodiments, thehaptic surface can be configured to macroscopically alter its physicalshape in response to a deformation haptic effect produced by actuator630. In these embodiments, actuator 630 can be a deformation actuatorthat can be configured to produce a deformation haptic effect. In othercertain embodiments, the haptic surface can be configured to produce amechanical impedance in response to an impedance haptic effect producedby actuator 630. In these embodiments, actuator 630 can be an impedanceactuator that can be configured to produce an impedance haptic effect.Thus, in certain embodiments, actuator 630 can cause housing 620 ofdevice 600 to reflect the information detected by sensor 610, includingthe extra-sensory information detected by sensor 610.

In one embodiment, sensor 610 can be a sensor configured to detect abarometric pressure, and actuator 630 can cause housing 620 to apply aforce, to deform, or to produce a mechanical impedance based on thedetected barometric pressure. Thus, in this embodiment, device 600 canindicate possible imminent changes in weather patterns due to anincrease or decrease in detected barometric pressure. In anotherembodiment, sensor 610 can be a magnetometer configured to detect anelectromagnetic field, and actuator 630 can cause housing 620 to apply aforce, to deform, or to produce a mechanical impedance based on one ormore properties of the detected electromagnetic field, such as thepresence of the electromagnetic field, the intensity of theelectromagnetic field, or the periodicity of the electromagnetic field.For example, a user could hold device 600 near an electrical cord todetermine whether the electrical cord is connected to a power source inresponse to housing 620 of device 600 applying a force, applying adeformation, or producing a mechanical impedance in the presence of analternating electromagnetic field surrounding the cord.

In another embodiment, sensor 610 can be a radiation sensor configuredto detect radiation, and actuator 630 can cause housing 620 to apply aforce, to deform, or to produce a mechanical impedance based on thedetected radiation. For example, device 600 can detect a level ofambient radiation within an environment using sensor 610, and actuator630 can cause housing 620 to apply a force, to deform, or to produce amechanical impedance based on the detected level of ambient radiation.In another embodiments, sensor 610 can be a sensor configured to detecta signal sent by a global positioning system (“GPS”), where the signalcan represent a position of device 600, and actuator 630 can causehousing 620 to apply a force, to deform, or to produce a mechanicalimpedance based on the detected signal (i.e., the detected position ofdevice 600). Thus, device 600 can create a geofence (i.e., a virtualperimeter for a real-world geographical area).

In another embodiment, sensor 610 can be a galvanic skin response sensorconfigured to detect an electrical conductance of the skin of a user,and actuator 630 can cause housing 620 to apply a force, to deform, orto produce a mechanical impedance based on characteristics of the useridentified from the detected electrical conductance of the skin of theuser, such as mood, ambient awareness, and bio feedback. Thus, device600 can apply a force, deform, or produce a mechanical impedance inorder to mirror an identified state of the user.

FIG. 7 illustrates a block diagram of a device 700 capable of producinga haptic effect to augment visual information displayed within a displayscreen, according to one embodiment of the invention. Device 700 caninclude sensor 710. Sensor 710 can be identical to sensor 28 of FIG. 1.Sensor 710 can be a sensor configured to detect information. Based onthe detected information, the sensor can generate sensor input, wherethe sensor input represents the detected information. In certainembodiments, sensor 710 can be a sensor configured to detectextra-sensory information, where extra-sensory information isinformation that cannot normally be perceived by a human being. Device700 can further include a display 720. Display 720 can be configured todisplay a visual image. Display 720 can be operatively coupled to anactuator 730. Display 720 can be configured to macroscopically alter itsphysical shape in response to a deformation haptic output effectproduced by actuator 730. In certain embodiments, actuator 730 can be adeformation actuator that can be configured to produce a deformationhaptic effect. Thus, in these embodiments, device 700 can be configuredto overlay a visual image displayed within display 720 with adeformation of at least a portion of a screen of display 720, where thedeformation is produced by actuator 730 and based on informationdetected by sensor 710. For example, display 720 can display a scene,and can further display invisible features of the scene as screensurface features/deformations within display 720. Thus, the visual scenedisplayed within 720 is generally not changed from how it actuallylooks.

In one embodiment, sensor 710 can be a light sensor configured to detectlight frequencies, including light frequencies that are within a rangeof the human eye, and also including light frequencies that are out ofrange of the human eye, and sensor 710 can be configured to capture afull-spectrum photograph of a scene. According to the embodiment, avisual image that represents the light frequencies that are within arange of the human eye can be displayed within display 720, and adeformation of at least a portion of a screen of display 720 thatrepresents the light frequencies that are out of range of the human eyecan be produced, where the deformation is produced by actuator 730 andbased on information detected by sensor 710. Thus, the visual image thatis displayed within display 720 is preserved, but the light frequenciesthat are out of range of the human eye detected by sensor 710 can beperceived by the user.

FIG. 8 illustrates a diagram of an example mapping of sensor input to aforce haptic signal that, when played at an actuator, produces a forcehaptic effect, according to one embodiment of the invention. Morespecifically, FIG. 8 illustrates sensor 800. Sensor 800 can be identicalto sensor 28 of FIG. 1. Sensor 800 can be a sensor configured to detectinformation. In certain embodiments, sensor 800 can be a sensorconfigured to detect extra-sensory information, where extra-sensoryinformation is information that cannot normally be perceived by a humanbeing. In other embodiments, sensor 800 can be a sensor configured todetect sensory information, where sensory information is informationthat can normally be perceived by a human being.

FIG. 8 also illustrates haptic representation module 810. When executedby a processor (not illustrated in FIG. 8), haptic representation module810 can receive the sensor input from sensor 800, map the sensor inputto a force haptic signal, and send the force haptic signal to actuator820. According to the embodiment, haptic representation module 810 caninclude an algorithm that maps the sensor input to a force hapticsignal, where the force haptic signal is configured to produce a forcehaptic effect when played at actuator 820. Thus, in certain embodiments,haptic representation module 810 can represent information (which caninclude sensory information or extra-sensory information) detected bysensor 800 as a force haptic effect that can be played at actuator 820.

In certain embodiments, sensor 800 can be local to haptic representationmodule 810 (i.e., sensor 800 and haptic representation module 810 can belocated within a single device). In other embodiments, sensor 800 islocated on a separate device from haptic representation module 810, andthe sensor input is sent to haptic representation module 810 over anetwork.

In certain embodiments, the mapping performed at haptic representationmodule 810 includes determining whether the received sensor inputexceeds a specified threshold (i.e., “specified value”). If the receivedsensor input exceeds the specified value, a corresponding force hapticsignal is generated, and the received sensor input is mapped to thegenerated force haptic signal. If the received sensor input does notexceed the specified value, no force haptic signal is generated.

In alternate embodiments, the mapping performed at haptic representationmodule 810 includes mathematically transforming the received sensorinput into a corresponding force haptic signal. This can be donecontinuously as sensor input is received, and thus, the correspondingforce haptic signal can be continuously modulated. As the force hapticsignal is continuously modulated, the corresponding force haptic effectthat is generated can also be continuously modulated.

FIG. 8 also illustrates actuator 820. Actuator 820 can be identical toactuator 26 of FIG. 1. Actuator 820 can receive the force haptic signalsent by haptic representation module 810, and can generate a forcehaptic effect based on the force haptic signal. In some embodiments,actuator 820 can be a force actuator configured to produce force hapticeffects.

Thus, in certain embodiments, actuator 820 can cause a device to apply astatic force (such as pressure) to a user of the device. For example,the device can apply a static force to a user as the user grips thedevice. As another example, the device can apply a static force to theuser while the device is in the user's pocket. The static force can beof a sufficient degree that the static force causes the device to deformor change shape. The user can feel the device deform by virtue of thedevice being in static contact with some part of the user's body. Insome embodiments, the device can apply a static force, such that thedevice deforms only to a degree that the deformation can be felt, butnot seen. In other embodiments, the device can apply a static force,such that the device deforms to a sufficient degree so that the user canvisually assess the state of the deformation as well. In certainembodiments, actuator 820 can cause a device to apply a dynamic force(such as pressure) to a user of the device, where the device continuallyapplies the force in a rhythmic or periodic way over a period of time.

FIG. 9 illustrates a diagram of an example mapping of sensor input to adeformation haptic signal that, when played at an actuator, produces adeformation haptic effect, according to one embodiment of the invention.More specifically, FIG. 9 illustrates sensor 900. Sensor 900 can beidentical to sensor 28 of FIG. 1. Sensor 900 can be a sensor configuredto detect information. In certain embodiments, sensor 900 can be asensor configured to detect extra-sensory information, whereextra-sensory information is information that cannot normally beperceived by a human being. In other embodiments, sensor 900 can be asensor configured to detect sensory information, where sensoryinformation is information that can normally be perceived by a humanbeing.

FIG. 9 also illustrates haptic representation module 910. When executedby a processor (not illustrated in FIG. 9), haptic representation module910 can receive the sensor input from sensor 900, map the sensor inputto a deformation haptic signal, and send the deformation haptic signalto actuator 920. According to the embodiment, haptic representationmodule 910 can include an algorithm that maps the sensor input to adeformation haptic signal, where the deformation haptic signal isconfigured to produce a deformation haptic effect when played atactuator 920. Thus, in certain embodiments, haptic representation module910 can represent information (which can include sensory information orextra-sensory information) detected by sensor 900 as a deformationhaptic effect that can be played at actuator 920.

In certain embodiments, sensor 900 can be local to haptic representationmodule 910 (i.e., sensor 900 and haptic representation module 910 can belocated within a single device). In other embodiments, sensor 900 islocated on a separate device from haptic representation module 910, andthe sensor input is sent to haptic representation module 910 over anetwork.

In certain embodiments, the mapping performed at haptic representationmodule 910 includes determining whether the received sensor inputexceeds a specified threshold (i.e., “specified value”). If the receivedsensor input exceeds the specified value, a corresponding deformationhaptic signal is generated, and the received sensor input is mapped tothe generated deformation haptic signal. If the received sensor inputdoes not exceed the specified value, no deformation haptic signal isgenerated.

In alternate embodiments, the mapping performed at haptic representationmodule 910 includes mathematically transforming the received sensorinput into a corresponding deformation haptic signal. This can be donecontinuously as sensor input is received, and thus, the correspondingdeformation haptic signal can be continuously modulated. As thedeformation haptic signal is continuously modulated, the correspondingdeformation haptic effect that is generated can also be continuouslymodulated.

FIG. 9 also illustrates actuator 920. Actuator 920 can be identical toactuator 26 of FIG. 1. Actuator 920 can receive the deformation hapticsignal sent by haptic representation module 910, and can generate adeformation haptic effect based on the deformation haptic signal. Insome embodiments, actuator 920 can be a deformation actuator configuredto produce deformation haptic effects.

Thus, in certain embodiments, actuator 920 can cause a device to deformor change shape. In some embodiments, the device can deform only to adegree that the deformation can be felt, but not seen. For example, ifthe device is in a pocket of a user, the user can reach into his pocketand feel the device and assess the shape of the device. In one example,if the housing of the device is flat, the user has not received anyvoicemail messages. However, if the housing of the device is extended(i.e., actuator 920 has caused the housing of the device to deform andextend out), then the user has received voicemail messages. In otherembodiments, the device can deform to a sufficient degree so that theuser can visually assess the state of the deformation as well. Incertain embodiments, actuator 920 can cause a device to continuallydeform in a rhythmic or periodic way over a period of time. For example,the sides of the device can pulse in a way that simulates a breathingpattern. In this example, the device can also display information aboutits internal state according to how hard or how fast it is “breathing”.As another example, the deformation of the device can simulate aheartbeat, and the device can display information about its internalstate depending on how hard or fast it is “beating.” Furthermore, thedevice can overlay one or more deformations on top of each other. As anexample, the device could deform in a manner to simulate a breathingpattern, and deform in a manner to simulate a heartbeat pattern,simultaneously, but the user would able to differentiate thesimultaneous patterns.

FIG. 10 illustrates a diagram of an example mapping of sensor input to aimpedance haptic signal that, when played at an actuator, produces animpedance haptic effect, according to one embodiment of the invention.More specifically, FIG. 10 illustrates sensor 1000. Sensor 1000 can beidentical to sensor 28 of FIG. 1. Sensor 1000 can be a sensor configuredto detect information, such as pressure applied to a device by a user.

FIG. 10 also illustrates haptic representation module 1010. Whenexecuted by a processor (not illustrated in FIG. 10), hapticrepresentation module 1010 can receive the sensor input from sensor1000, map the sensor input to an impedance haptic signal, and send theimpedance haptic signal to actuator 1020. According to the embodiment,haptic representation module 1010 can include an algorithm that maps thesensor input to an impedance haptic signal, where the impedance hapticsignal is configured to produce an impedance haptic effect when playedat actuator 1020. Thus, in certain embodiments, haptic representationmodule 1010 can represent information (which can include sensoryinformation or extra-sensory information) detected by sensor 1000 as animpedance haptic effect that can be played at actuator 1020.

In certain embodiments, sensor 1000 can be local to hapticrepresentation module 1010 (i.e., sensor 1000 and haptic representationmodule 1010 can be located within a single device). In otherembodiments, sensor 1000 is located on a separate device from hapticrepresentation module 1010, and the sensor input is sent to hapticrepresentation module 1010 over a network.

In certain embodiments, the mapping performed at haptic representationmodule 1010 includes determining whether the received sensor inputexceeds a specified threshold (i.e., “specified value”). If the receivedsensor input exceeds the specified value, a corresponding impedancehaptic signal is generated, and the received sensor input is mapped tothe generated impedance haptic signal. If the received sensor input doesnot exceed the specified value, no impedance haptic signal is generated.

FIG. 10 also illustrates actuator 1020. Actuator 1020 can be identicalto actuator 26 of FIG. 1. Actuator 1020 can receive the impedance hapticsignal sent by haptic representation module 1010, and can generate animpedance haptic effect based on the impedance haptic signal. In someembodiments, actuator 1020 can be an impedance actuator configured toproduce impedance haptic effects.

Thus, in certain embodiments, actuator 1020 can cause a device toproduce an impedance haptic effect in response to user input, such aspressure applied to the device by the user. For example, a user may“query” the device to find out a shape of the device, by hapticallyexploring the contours of the device with a hand or finger. In anotherexample, the user squeezes the device to feel the impedance ordeformability. In this way, actuator 1020 can cause the housing of thedevice to display information to the user, not in a form of deformation,but in a form of mechanical impedance. For example, the user can squeezethe device, and if the device feels “soft,” then the user does not haveany urgent voicemail messages. However, the user can also squeeze thedevice, and if the device feels “hard,” then the user does have urgentvoicemail messages. Thus, the information can be queried by a user in adiscreet, intuitive way by applying pressure to the device andascertaining the amount of mechanical impedance felt.

While the embodiments described above have involved individual forcehaptic effects, deformation haptic effects, or impedance haptic effects,in alternate embodiments, an actuator can cause a device to producevarious combinations of force haptic effects, deformation effects, andimpedance haptic effects, as well as vibrotactile effects or any otherhaptic effects.

FIG. 11 illustrates a flow diagram of the functionality of a hapticrepresentation module (such as haptic representation module 16 of FIG.1), according to one embodiment of the invention. The flow begins andproceeds to 1110. At 1110, input is received from a sensor. The inputcan include extra-sensory information, where extra-sensory informationis information that cannot normally be perceived by a human being. Incertain embodiments, the input can include one or more interactionparameters, where each interaction parameter can include a value.

Also, in certain embodiments, the sensor can be a sensor configured todetect a barometric pressure. In other embodiments, the sensor can be amagnetometer configured to detect an electromagnetic field. In otherembodiments, the sensor can be a radiation sensor configured to detectradiation. In other embodiments, the sensor can be a sensor configuredto detect a signal sent by a GPS, where the signal can represent aposition of a device. In other embodiments, the sensor can be a galvanicskin response sensor configured to detect an electrical conductance ofthe skin of a user. In other embodiments, the sensor can be a lightsensor configured to detect light frequencies, including lightfrequencies that are within a range of the human eye, and also includinglight frequencies that are out of range of the human eye. The flowproceeds to 1120.

At 1120, the received input is mapped to a haptic signal. In certainembodiments, the mapping the received input to the haptic signalincludes generating the haptic signal when the value of at least oneinteraction parameter exceeds a specified value. In other embodiments,the mapping the received input to the haptic signal includescontinuously modulating the haptic signal based on a continuous updatingof the value of at least one interaction parameter. The haptic signalcan be a force haptic signal. The haptic signal can be a deformationhaptic signal. The haptic signal can be an impedance haptic signal. Thehaptic signal can be any combination of a force haptic signal, adeformation haptic signal, or an impedance haptic signal. The flowproceeds to 1130.

At 1130, the haptic signal is sent to an actuator to generate a hapticeffect. In embodiments where the haptic signal is a force haptic signal,the haptic effect is a force haptic effect. In embodiments where thehaptic signal is a deformation haptic signal, the haptic effect is adeformation haptic effect. In embodiments where the haptic signal is animpedance haptic signal, the haptic effect is an impedance hapticeffect. In some embodiments where the haptic effect is a force hapticeffect, the force haptic effect can cause a device to apply a staticforce to a user. In other embodiments where the haptic effect is a forcehaptic effect, the force haptic effect can cause the device to apply adynamic force to the user. In certain embodiments where the hapticeffect is a force haptic effect, the force haptic effect can cause thedevice to apply pressure to the user's body. In certain embodimentswhere the haptic effect is a deformation haptic effect, the deformationhaptic effect can cause a device to deform. In other embodiments wherethe haptic effect is a deformation haptic effect, the deformation hapticeffect can cause the device to continuously deform over a period oftime. In certain embodiments where the haptic effect is a deformationhaptic effect, the deformation haptic effect can cause the device tomodify at least one of a macro-shape of the device or a texture of thedevice, and the deformation of the devices can comprise at least one ofa visual deformation, a haptic deformation, a quasi-static deformation,or a dynamic deformation. In embodiments where the haptic effect is animpedance haptic effect, the impedance haptic effect can cause a deviceto produce a mechanical impedance in response to user input. In certainembodiments where the haptic effect is an impedance haptic effect, theimpedance haptic effect can cause the device to stiffen in response topressure that is applied to the device by a user.

In embodiments where the sensor is a sensor configured to detect abarometric pressure, the haptic effect that is generated by the actuatorcan cause a device to deform based on the detected barometric pressure.In embodiments where the sensor is a magnetometer configured to detectan electromagnetic field, the haptic effect that is generated by theactuator can cause a device to deform based on one or more properties ofthe detected electromagnetic field. In embodiments where the sensor is aradiation sensor configured to detect radiation, the haptic effect thatis generated by the actuator can cause a device to deform based on thedetected radiation. In embodiments where the sensor is a sensorconfigured to detect a signal sent by a GPS, where the signal representsa position of a device, the haptic effect that is generated by theactuator can cause the device to deform based on the detected signal. Inembodiments where the sensor is a galvanic skin response sensorconfigured to detect an electrical conductance of the skin of a user,the haptic effect that is generated by the actuator can cause the deviceto deform based on one or more characteristics of the user identifiedfrom the detected electrical conductance of the skin of the user. Inembodiments where the sensor is a light sensor configured to detectlight frequencies, including light frequencies that are within a rangeof the human eye, and also including light frequencies that are out ofrange of the human eye, the haptic effect that is generated by theactuator can cause a display of the device to deform based on thedetected light frequencies that are out of range of the human eye. Theflow then ends.

Thus, a haptic representation system is provided that represents sensorinput as one or more haptic effects. The haptic representation systemcan enable richer, more informative, and more interactive experiencesfor a user. The haptic representation system has wide ranging use cases,such as military, medical, automotive, and mobility.

The features, structures, or characteristics of the invention describedthroughout this specification may be combined in any suitable manner inone or more embodiments. For example, the usage of “one embodiment,”“some embodiments,” “certain embodiment,” “certain embodiments,” orother similar language, throughout this specification refers to the factthat a particular feature, structure, or characteristic described inconnection with the embodiment may be included in at least oneembodiment of the present invention. Thus, appearances of the phrases“one embodiment,” “some embodiments,” “a certain embodiment,” “certainembodiments,” or other similar language, throughout this specificationdo not necessarily all refer to the same group of embodiments, and thedescribed features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

One having ordinary skill in the art will readily understand that theinvention as discussed above may be practiced with steps in a differentorder, and/or with elements in configurations which are different thanthose which are disclosed. Therefore, although the invention has beendescribed based upon these preferred embodiments, it would be apparentto those of skill in the art that certain modifications, variations, andalternative constructions would be apparent, while remaining within thespirit and scope of the invention. In order to determine the metes andbounds of the invention, therefore, reference should be made to theappended claims.

1. (canceled)
 2. A handheld device, comprising: a housing including ahaptic surface having a shape, the haptic surface configured to performat least one of applying a force to generate a force haptic effect,deforming by altering the shape to generate a deformation haptic effect,or producing a mechanical impedance to generate an impedance hapticeffect; at least one sensor configured to detect extra-sensoryinformation; at least one haptic output device coupled to the hapticsurface; and a processor, coupled to the at least one sensor and thehaptic output device, the processor configured to: receive theextra-sensory information from the at least one sensor, map theextra-sensory information to a haptic signal, and send the haptic signalto the haptic output device to generate at least one of the force hapticeffect, the deformation haptic effect, or the impedance haptic effect.3. The handheld device of claim 2, wherein the at least one sensorincludes a pressure sensor configured to detect a barometric pressure,and the haptic surface is configured to perform at least one of applyingthe force, deforming, or producing the mechanical impedance based on anincrease or decrease of the barometric pressure.
 4. The handheld deviceof claim 2, wherein: the at least one sensor includes a magnetometerconfigured to detect an electromagnetic field; the haptic surface isconfigured to perform at least one of applying the force, deforming, orproducing the mechanical impedance based on one or more properties ofthe electromagnetic field; and the properties of the electromagneticfield include a presence of at least one of the electromagnetic field,an intensity of the electromagnetic field or a periodicity of theelectromagnetic field.
 5. The handheld device of claim 2, wherein the atleast one sensor includes a radiation sensor configured to detectambient radiation, and the haptic surface is configured to perform atleast one of applying the force, deforming, or producing the mechanicalimpedance based on a level of the ambient radiation.
 6. The handhelddevice of claim 2, wherein the at least one sensor includes a globalpositioning system (GPS) receiver configured to receive a GPS signal anddetermine a position of the handheld device, and the haptic surface isconfigured to perform at least one of applying the force, deforming, orproducing the mechanical impedance based on the position of the handhelddevice.
 7. The handheld device of claim 6, wherein the haptic signal issent to a plurality of haptic devices coupled to the haptic surface tocause the haptic surface to deform at different locations on the housingto provide directional information.
 8. The handheld device of claim 6,wherein the processor is further configured to create a virtualperimeter for a geographic area based on the position.
 9. The handhelddevice of claim 2, wherein the at least one sensor includes a galvanicskin response sensor configured to detect an electrical conductance ofthe skin of a user, and the haptic surface is configured to perform atleast one of applying the force, deforming, or producing the mechanicalimpedance based on characteristics of the user identified from theelectrical conductance of the skin of the user.
 10. The handheld deviceof claim 9, wherein the characteristics of the user include a mood, anambient awareness or a bio feedback.
 11. The handheld device of claim 2,wherein: the extra-sensory information is continuously received; andmapping the extra-sensory information to the haptic signal includescontinuously modulating the haptic signal based on a continuous updatingof the extra-sensory information.
 12. A method of generating a hapticeffect on a handheld device, the method comprising: receivingextra-sensory information from a sensor; mapping the extra-sensoryinformation to a haptic signal; and sending the haptic signal to ahaptic output device, coupled to a haptic surface having a shape, tocause the haptic surface to generate at least one of a force hapticeffect by applying a force, a deformation haptic effect by altering theshape, or an impedance haptic effect by producing a mechanicalimpedance.
 13. The method of claim 12, wherein the extra-sensoryinformation includes a barometric pressure detected by the sensor, andthe haptic signal is configured to cause the haptic surface to generateat least one of the force haptic effect, the deformation haptic effect,or the impedance haptic effect based on an increase or decrease of thebarometric pressure.
 14. The method of claim 12, wherein: theextra-sensory information includes one or more properties of anelectromagnetic field detected by the sensor; the haptic signal isconfigured to cause the haptic surface to generate at least one of theforce haptic effect, the deformation haptic effect or the impedancehaptic effect based on the one or more properties of the electromagneticfield; and the one or more properties of the electromagnetic fieldinclude at least one of a presence of the electromagnetic field, anintensity of the electromagnetic field or a periodicity of theelectromagnetic field.
 15. The method of claim 12, wherein theextra-sensory information includes ambient radiation detected by thesensor, and the haptic signal is configured to cause the haptic surfaceto generate at least one of the force haptic effect, the deformationhaptic effect, or the impedance haptic effect based on a level of theambient radiation.
 16. The method of claim 12, wherein the extra-sensoryinformation includes a global positioning system (GPS) signal receivedby the sensor and used to determine a position of the handheld device,and the haptic signal is configured to cause the haptic surface togenerate at least one of the force haptic effect, the deformation hapticeffect, or the impedance haptic effect based on the position of thehandheld device.
 17. The method of claim 16, wherein the haptic signalis sent to a plurality of haptic devices coupled to the haptic surfaceto cause the haptic surface to deform at different locations on ahousing of the handheld device to provide directional information. 18.The method of claim 16, further comprising: creating a virtual perimeterfor a geographic area based on the position.
 19. The method of claim 12,wherein the extra-sensory information includes an electrical conductanceof the skin of a user detected by the sensor, and the haptic signal isconfigured to cause the haptic surface to generate at least one of theforce haptic effect, the deformation haptic effect, or the impedancehaptic effect based on characteristics of the user identified from theelectrical conductance of the skin of the user.
 20. The method of claim19, wherein the characteristics of the user include a mood, an ambientawareness or a bio feedback.
 21. The method of claim 12, wherein: theextra-sensory information is continuously received; and the mapping theextra-sensory information to the haptic signal includes continuouslymodulating the haptic signal based on a continuous updating of theextra-sensory information.